Method for using an expander to connect to different storage interconnect architectures

ABSTRACT

Provided is a system for interfacing with storage units, including a backplane, at least one slot in the storage enclosure for receiving one storage unit, and two physical interfaces on the backplane for at least one slot. The storage unit is capable of being positioned in the slot to mate with one of the two physical interfaces for the slot, wherein each physical interface supports different storage interconnect architectures.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to the following and commonly assignedpatent applications filed on the same date hereof:

“An Adaptor Supporting Different Protocols”, by Pak-Lung Seto and DeifAtallah, having U.S. application Ser. No. 10/742,029; and

“Enclosure Management Device”, by Pak-Lung Set, having U.S. applicationSer. No. 10/742,030.

This application is a divisional of application Ser. No. 10/741,460, nowU.S. Pat. No. 7,155,546 filed on Dec. 18, 2003 and issued Dec. 26, 2006,which patent application is incorporated herein in its entirety.

BACKGROUND

1. Field

The present embodiments relate to a method, system, and program forsupporting multiple interfaces in a storage enclosure.

2. Description of the Related Art

An adaptor or multi-channel protocol controller enables a device coupledto the adaptor to communicate with one or more connected end devicesaccording to a storage interconnect architecture, also known as ahardware interface, where a storage interconnect architecture defines astandard way to communicate and recognize such communications, such asSerial Attached Small Computer System Interface (SCSI) (SAS), SerialAdvanced Technology Attachment (SATA), Fibre Channel, etc. These storageinterconnect architectures allow a device to maintain one or moreconnections to another end device via a point-to-point connection, anarbitrated loop of devices, an expander providing a connection tofurther end devices, or a fabric comprising interconnected switchesproviding connections to multiple end devices. In the SAS/SATAarchitecture, a SAS port is comprised of one or more SAS PHYs, whereeach SAS PHY interfaces a physical layer, i.e., the physical interfaceor connection, and a SAS link layer having multiple protocol link layer.Communications from the SAS PHYs in a port are processed by thetransport layers for that port. There is one transport layer for eachSAS port to interface with each type of application layer supported bythe port. A “PHY” as defined in the SAS protocol is a device object thatis used to interface to other devices and a physical interface. Furtherdetails on the SAS architecture for devices and expanders is describedin the technology specification “Information Technology—Serial AttachedSCSI (SAS)”, reference no. ISO/IEC 14776-150:200x and ANSIINCITS.***:200x PHY layer (Jul. 9, 2003), published by ANSI; details onthe Fibre Channel architecture are described in the technologyspecification “Fibre Channel Framing and Signaling Interface”, documentno. ISO/IEC AWI 14165-25; details on the SATA architecture are describedin the technology specification “Serial ATA: High Speed Serialized ATAttachment” Rev. 1.0A (January 2003).

Within an adaptor, the PHY layer performs the serial to parallelconversion of data, so that parallel data is transmitted to layers abovethe PHY layer, and serial data is transmitted from the PHY layer throughthe physical interface to the PHY layer of a receiving device. In theSAS specification, there is one set of link layers for each SAS PHYlayer, so that effectively each link layer protocol engine is coupled toa parallel-to-serial converter in the PHY layer. A connection pathconnects to a port coupled to each PHY layer in the adaptor andterminate in a physical interface within another device or on anexpander device, where the connection path may comprise a cable oretched paths on a printed circuit board.

An expander is a device that facilitates communication and provides forrouting among multiple SAS devices, where multiple SAS devices andadditional expanders connect to the ports on the expander, where eachport has one or more SAS PHYs and corresponding physical interfaces. Theexpander also extends the distance of the connection between SASdevices. The expander may route information from a device connecting toa SAS PHY on the expander to another SAS device connecting to theexpander PHYs. In SAS, using the expander requires additional serial toparallel conversions in the PRY layers of the expander ports. Uponreceiving a frame, a serial-to-parallel converter, which may be part ofthe PHY, converts the received data from serial to parallel to routeinternally to an output SAS PHY, which converts the frame from parallelto serial to the target device. The SAS PHY may convert parallel data toserial data through one or more encoders and convert serial data toparallel data through a parallel data builder and one or more decoders.A phased lock loop (PLL) may be used to track incoming serial data andlock into the frequency and phase of the signal. This tracking of thesignal may introduce noise and error into the signal.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings in which like reference numbers representcorresponding parts throughout:

FIGS. 1 and 2 illustrate a system and adaptor in accordance withembodiments;

FIGS. 3, 4, and 5 illustrate operations implemented in the adaptor ofFIGS. 1 and 2;

FIG. 6 illustrates a perspective view of a storage enclosure inaccordance with embodiments;

FIG. 7 illustrates a storage enclosure backplane and attached storageserver in accordance with embodiments;

FIG. 8 illustrates an expander PHY in accordance with embodiments;

FIG. 9 illustrates a front view of a rack including storage enclosuresand servers in accordance with embodiments;

FIG. 10 illustrates an adaptor that may be used with the storage serverin FIG. 7 in accordance with embodiments; and

FIG. 11 illustrates system components that may be used with thedescribed embodiments.

DETAILED DESCRIPTION

In the following description, reference is made to the accompanyingdrawings which form a part hereof and which illustrate severalembodiments. It is understood that other embodiments may be utilized andstructural and operational changes may be made.

Supporting Multiple Storage Interconnect Architectures in an Adaptor

FIG. 1 illustrates a computing environment in which embodiments may beimplemented. A host system 2 includes one or more central processingunits (CPU) 4 (only one is shown), a volatile memory 6, non-volatilestorage 8, an operating system 10, and one or more adaptors 12 a, 12 bwhich maintains physical interfaces to connect with other end devicesdirectly in a point-to-point connection or indirectly through one ormore expanders, one or more switches in a fabric or one or more devicesin an arbitrated loop. An application program 16 further executes inmemory 6 and is capable of transmitting to and receiving informationfrom the target device through one of the physical interfaces in theadaptors 12 a, 12 b. The host 2 may comprise any computing device knownin the art, such as a mainframe, server, personal computer, workstation,laptop, handheld computer, telephony device, network appliance,virtualization device, storage controller, etc. Various CPUs 4 andoperating system 10 known in the art may be used. Programs and data inmemory 6 may be swapped into storage 8 as part of memory managementoperations.

The operating system 10 may load a device driver 20 a, 20 b, 20 c foreach protocol supported in the adaptor 12 a, 12 b to enablecommunication with a device communicating using the supported protocoland also load a bus driver 24, such as a Peripheral ComponentInterconnect (PCI) interface, to enable communication with a bus 26.Further details of PCI interface are described in the publication “PCILocal Bus, Rev. 2.3”, published by the PCI-SIG. The operating system 10may load device drivers 20 a, 20 b, 20 c supported by the adaptors 12 a,12 b upon detecting the presence of the adaptors 12 a, 12 b, which mayoccur during initialization or dynamically, such as the case withplug-and-play device initialization. In the embodiment of FIG. 1, theoperating system 10 loads three protocol device drivers 20 a, 20 b, 20c. For instance, the device drivers 20 a, 20 b, 20 c may support theSAS, SATA, and Fibre Channel point-to-point storage interfaces, i.e.,interconnect architectures. Additional or fewer device drivers may beloaded based on the number of device drivers the adaptor 12 supports.

FIG. 2 illustrates an embodiment of adaptor 12, which may comprise theadaptors 12 a, 12 b. Each adaptor includes a plurality of physicalinterfaces 30 a, 30 b . . . 30 n, which may include the transmitter andreceiver circuitry and other connection hardware. The physical interfacemay connect to another device via cables or a path etched on a printedcircuit board so that devices on the printed circuit board communicatevia etched paths. The physical interfaces 30 a, 30 b . . . 30 n mayprovide different physical interfaces for different device connections,such as one physical interface 30 a, 30 b . . . 30 n for connecting to aSAS/SATA device and another interface for a Fibre Channel device. Eachphysical interface 30 a, 30 b . . . 30 n may be coupled to a PHY layer32 a, 32 b . . . 32 n within expander 34. The PHY layer 32 a, 32 b . . .32 n provides for an encoding scheme, such as 8 b 10 b, to translatebits, and a clocking mechanism, such as a phased lock loop (PLL). ThePHY layer 32 a, 32 b . . . 32 n would include a serial-to-parallelconverter to perform the serial-to-parallel conversion and the PLL totrack the incoming data and provide the data clock of the incoming datato the serial-to-parallel converter to use when performing theconversion. Data is received at the adaptor 12 in a serial format, andis converted at the SAS PHY layer 32 a, 32 b . . . 32 n to the parallelformat for transmission within the adaptor 12. The SAS PHY layer 32 a,32 b . . . 32 n further provides for error detection, bit shift andamplitude reduction, and the out-of-band (OOB) signaling to establish anoperational link with another SAS PHY in another device. The terminterface may refer to the physical interface or the interfaceperforming operations on the received data implemented as circuitry, orboth.

The PHY layer 32 a, 32 b . . . 32 n further performs the speednegotiation with the PHY in the external device transmitting data toadaptor 12. In certain embodiments, the PHY layer 32 a, 32 b . . . 32 nmay be programmed to allow speed negotiation and detection of differentprotocols transmitting at the same or different transmission speeds. Forinstance, SATA and SAS transmissions can be detected because they aretransmitted at speeds of 1.5 gigahertz (GHz) and 3 GHz and Fibre Channeltransmissions can be detected because they are transmitted at 1.0625GHz, 2.125 GHz, and 4.25 GHz. Because link transmission speeds may bedifferent for certain storage interfaces, the PHY layer 32 a, 32 b . . .32 n may detect storage interfaces having different link speeds bymaintaining information on speeds for different storage interfaces.However, certain different storage interfaces, such as SAS and SATA, maytransmit at the same link speeds and support common transport protocols.If storage interfaces transmit at a same link speed, then the PHY layer32 a, 32 b . . . 32 n may distinguish among storage interfaces capableof transmitting at the same speed by checking the transmission format todetermine the storage interface and protocol, where the link protocoldefines the characteristics of the transmission, including speed andtransmission data format.

For instance, the SAS and SATA protocol can be distinguished not only bytheir transmission speeds, but also by their use of the OOB signal.Other protocols, such as Fibre Channel do not use the OOB signal. FibreChannel, SAS and SATA all have a four byte primitive. The primitive ofSATA can be distinguished because the first byte of the SATA primitiveindicates “K28.3”, whereas the first byte of the SAS and Fibre Channelprimitive indicates “K28.5”. The SAS and Fibre Channel primitives can bedistinguished based on the content of the next three bytes of theirprimitives, which differ. Thus, the content of the primitives can beused to distinguish between the SAS, SATA and Fibre Channel protocols.Additionally, different of the protocols, such as SAS and Fibre Channelhave different handshaking protocols. Thus, the handshaking protocolbeing used by the device transmitting the information can be used todistinguish the storage connect interface being used.

The PHY layer 32 a, 32 b . . . 32 n forwards the frame to the link layer36 in the expander 34. The link layer 36 may maintain a set of elementsfor each protocol supported by a port, such as a Serial SCSI Protocol(SSP) link layer 38 to process SSP frames, a Serial Tunneling Protocol(STP) layer 38 b, a Serial Management Protocol (SMP) layer 38 c, and aFibre Channel link layer 38 d to support the Fibre Channel protocol fortransporting the frames. Within the expander 34, information is routedfrom one PHY to another. The transmitted information may includeprimitives, packets, frames, etc., and may be used to establish theconnection and open the address frame. A router 40 routes transmissionsbetween the protocol engines 42 a, 42 b and the PHY layers 32 a, 32 b .. . 32 n. The router 40 maintains a router table 41 providing anassociation of PHY layers 32 a, 32 b . . . 32 n to protocol engines 42a, 42 b, such that a transmission from a PHY layer or protocol engine isrouted to the corresponding protocol engine or PHY layer, respectively,indicated in the router table 41. If the protocol engines 42 a, 42 bsupport the transport protocol, e.g., SSP, STP, SMP, Fibre Channelprotocol, etc., associated with the link layer 38 a, 38 b, 83 c, 38 dforwarding the transmission, then the router 40 may use any techniqueknown in the art to select among the multiple protocol engines 42 a, 42b to process the transmission, such as round robin, load balancing basedon protocol engine 42 a, 42 b utilization, etc. The Fibre ChannelProtocol comprises the transport layer for handling informationtransmitted on a Fibre Channel storage interface. Data may becommunicated in frames, packets, primitives or any other datatransmission format known in the art. A transport layer comprises anycircuitry, including software or hardware, that is use to provide avirtual error-free, point to point connection to allow for thetransmission of information between devices so that transmittedinformation arrives un-corrupted and in the correct order. The transportlayer further establishes, e.g., opens, and dissolves connectionsbetween devices.

A transport protocol provides a set of transmission rules andhandshaking procedures used to implement a transport layer, oftendefined by an industry standard, such as SAS, SATA, Fibre Channel, etc.The transport layer and protocol may comprise those transport protocolsdescribed herein and others known in the art. The protocol engine 42 a,42 b comprises the hardware and/or software that implements differenttransport protocols to provide transport layer functionality fordifferent protocols.

Each protocol engine 42 a, 42 b is capable of performing protocolrelated operations for all the protocols supported by the adaptor 12.Alternatively, different protocol engines may support differentprotocols. For instance, protocol engine 42 b may support the sametransport layers as protocol engine 42 a or a different set of transportlayers. Each protocol engine 42 a, 42 b implements a port layer 44, anda transport layer, such as a SSP transport layer 46 a, STP transportlayer 46 b, SMP transport layer 46 c, and a Fibre Channel Protocoltransport layer 46 d. Further, the protocol engines 30 a, 30 b maysupport the transport and network layer related operations for thesupported protocols. The port layer 44 interfaces between the linklayers 38 a, 38 b, 38 c, 38 d via the router 40 and the transport layers46 a, 46 b, 46 c, 46 d to transmit information to the correct transportlayer or link layer. The PHYs 32 a, 32 b . . . 32 n and correspondingphysical interfaces 30 a, 30 b . . . 30 n may be organized into one ormore ports, where each SAS port has a unique SAS address. The portcomprises a component or construct to which interfaces are assigned. Anaddress comprises any identifier used to identify a device or component.The protocol engines 42 a, 42 b may further include one or more virtualPRY layers to enable communication with virtual PHY layers in the router40. A virtual PHY is an internal PHY that connects to another PHY insideof the device, and not to an external PHY. Data transmitted to thevirtual PHY typically does not need to go through a serial-to-parallelconversion.

Each protocol engine 42 a, 42 b includes an instance of the protocoltransport layers 46 a, 46 b, 46 c, 46 d, where there is one transportlayer to interface with each type of application layer 48 a, 48 b, 48 cin the application layer 50. The application layer 50 may be supportedin the adaptor 12 or host system 2 and provides network services to theend users. For instance, the SSP transport layer 46 a and Fibre ChannelProtocol (FCP) transport layer 46 b interface with a SCSI applicationlayer 48 a, the STP transport layer 46 c interfaces with an AdvancedTechnology Attachment (ATA) application layer 48 b, and the SMPtransport layer 46 d interfaces with a management application layer 48c. Further details of the ATA technology are described in thepublication “Information Technology—AT Attachment with PacketInterface—6 (ATA/ATAPI-6)”, reference no. ANSI INCITS 361-2002(September, 2002).

All the PHY layers 32 a, 32 b . . . 32 n may share the same link layerand protocol link layers, or there may be a separate instance of eachlink layer and link layer protocol 38 a, 38 b, 38 c, 38 d for each PHY.Further, each protocol engine 42 a, 42 b may include one port layer 44for all ports including the PHY layers 32 a, 32 b . . . 32 n or mayinclude a separate instance of the port layer 44 for each port in whichone or more PHY layers and the corresponding physical interfaces areorganized. Further details on the operations of the physical layer, PHYlayer, link layer, port layer, transport layer, and application layerand components implementing such layers described herein are found inthe technology specification “Information Technology—Serial AttachedSCSI (SAS)”, referenced above.

The router 40 allows the protocol engines 42 a, 42 b to communicate toany of the PHY layers 32 a, 32 b . . . 32 n. The protocol engines 42 a,42 b communicate parallel data to the PHY layers 32 a, 32 b . . . 32 n,which include parallel-to-serial converters to convert the parallel datato serial data for transmittal through the corresponding physicalinterface 30 a, 30 b . . . 30 n. The data may be communicated to a PHYon the target device or an intervening external expander. A targetdevice is a device to which information is transmitted from a source orinitiator device attempting to communicate with the target device.

With the described embodiments of FIGS. 1 and 2, one protocol engine 42a, 42 b having the port and transport layers can manage transmissions tomultiple PHY layers 32 a, 32 b . . . 32 n. The transport layers 46 a, 46b, 46 c, 46 d of the protocol engines 42 a, 42 b may only engage withone open connection at a time. However, if delays are experienced fromthe target on one open connection, the protocol engine 42 a, 42 b candisconnect and establish another connect to process I/O requests fromthat other connection to avoid latency delays for those target devicestrying to establish a connection. This embodiment provides greaterutilization of the protocol engine bandwidth by allowing each protocolengine to multiplex among multiple target devices and switch amongconnections. The protocol engines 42 a, 42 b and physical interface havegreater bandwidth than the target device, so that the target devicethroughput is lower than the protocol engine 42 a, 42 b throughput. Incertain embodiments, the protocol engines 42 a, 42 b may multiplexbetween different PHYs 32 a, 32 b . . . 32 n to manage multiple targets.

Allowing one protocol engine to handle multiple targets further reducesthe number of protocol engines that need to be implemented in theadaptor to support all the targets.

FIG. 3 illustrates operations performed by the PHY layers 32 a, 32 b . .. 32 n and the link layer 36 to open a connection with an initiatingdevice, where the initiating device may transmit using SAS, FibreChannel, or some other storage interface (storage interconnectarchitecture). The operation to establish the connection may occur afterthe devices are discovered during identification and linkinitialization. In response to a reset or power-on sequence, the PHYlayer 32 a, 32 b may begin (at block 100) link initialization byreceiving link initialization information, such as primitives, from aninitiator device at one physical interface 30 a, 30 b . . . 30 n (FIG.2). The PHY layer 32 a, 32 b . . . 32 n coupled to the receivingphysical interface 30 a, 30 b . . . 30 n performs (at block 102) speednegotiation to ensure that the link operates at the highest frequency.In certain embodiments, the PHY layer 32 a, 32 b . . . 32 n includes thecapability to detect and negotiate speeds for different storageinterfaces, where the different storage interfaces have differenttransmission characteristics, such as different transmission speedsand/or transmission information, such as is the case with the SAS/SATAand Fibre Channel storage interfaces. The PHY layer 32 a, 32 b . . . 32n then determines (at block 104) the storage interface used for thetransmission to establish the connection, which may be determined fromthe transmission speed if a unique transmission speed is associated witha storage interface or from characteristics of the transmission, such asinformation in the header of the transmission, format of thetransmission, etc. The PHY layer 32 a, 32 b forwards (at block 106) theinformation to the link layer 36 indicating which detected storageinterface to use (SAS/SATA or Fibre Channel).

If (at block 108) the determined storage interface complies with theSATA protocol, then the connection is established (at block 110) and nofurther action is necessary. If (at block 108) the connection utilizesthe SAS protocol, then the link layer 36 processes (at block 112) anOPEN frame to determine the SAS transport protocol to use (e.g., SSP,STP, SMP, Fibre Channel Protocol). The OPEN frame is then forwarded (atblock 114) to the determined SAS protocol link layer 38 a, 38 b, 38 c,38 d (SSP, STP, SMP, Fibre Channel Protocol) to process. The protocollink layer 38 a, 38 b, 38 c, 38 d then establishes (at block 116) anopen connection for all subsequent frames transmitted as part of thatopened connection. The connection must be opened using the OPEN framebetween an initiator and target port before communication may begin. Aconnection is established between one SAS initiator PHY in the SASinitiator port and one SAS target PHY in the SAS target port. If (atblocks 108 and 118) the storage interface complies with a point-to-pointFibre Channel protocol, then the connection is established (at block120). Otherwise, if (at blocks 108 and 118) the storage interfacecomplies with the Fibre Channel Arbitrated Loop protocol, then the FibreChannel link layer 38 d establishes (at block 122) the open connectionfor all subsequent frames transmitted as part of connection. The FibreChannel link layer 38 d may establish the connection using Fibre Channelopen primitives. Further details of the Fibre Channel Arbitrated Loopprotocol are described in the publication “Information Technology—FibreChannel Arbitrated Loop (FC-AL-2)”, having document no. ANSI INCITS332-1999.

With the described implementations, the PHY layer 32 a, 32 b . . . 32 nis able to determine the storage interface for different storageinterfaces that transmit at different transmission link speeds and/orhave different transmission characteristics. This determined storageinterface information is then forwarded to the link layer 36 to use todetermine which link layer protocol and transport protocol to use toestablish the connection, such as a SAS link layer protocol, e.g., 38 a,38 b, 38 c, or the Fibre Channel link layer protocol 38 d, where thedifferent protocols that may be used require different processing tohandle.

FIG. 4 illustrates operations performed by the router 40 to select aprotocol engine 42 a, 42 b to process the received frame. Upon receiving(at block 150) a transmission from the protocol link layer 38 a, 38 b,38 c, 38 d, such as a frame, packet, primitive, etc., to establish aconnection, if (at block 152) a router table 41 provides an associationof a protocol engine 42 a, 42 b for the PHY 32 a, 32 b . . . 32 nforwarding the transmission, then the router 40 forwards (at block 154)the transmission to the protocol engine 42 a, 42 b associated with thePHY indicated in the router table 41. If (at block 152) the router table41 does not provide an association of a PHY layer and protocol engineand if (at block 156) the protocol of the transmission complies with theSATA or Fibre Channel point-to-point protocol, then the router 40selects (at block 158) one protocol engine to use based on a selectioncriteria, such as load balancing, round robin, etc. If (at block 160)all protocol engines 46 a, 46 b capable of handling the determinedprotocol are busy, then fail is returned (at block 162) to the devicethat sent a transmission. Otherwise, if (at block 160) a protocol engine46 a, 46 b is available, then one protocol engine 46 a, 46 b is selected(at block 164) to use for the transmission and the transmission isforwarded to the selected protocol engine.

If (at block 156) the protocol of the connection request complies withthe SAS or Fibre Channel Arbitrated Loop protocol, then the router 40selects (at block 166) one protocol engine 46 a, 46 b to use based on aselection criteria. If (at block 168) all protocol engines 46 a, 46 bcapable of handling the determined protocol are busy, then the PHYreceiving the transmission is signaled that the connection requestfailed, and the PHY 32 a, 32 b . . . 32 n returns (at block 170) an OPENreject command to the transmitting device. Otherwise, if (at block 168)a protocol engine 46 a, 46 b is available, then an entry is added (atblock 172) to the router table 41 associating the PHY 42 a, 42 b . . .42 n forwarding the transmission with one protocol engine 46 a, 46 b.The router 40 signals (at block 174) the PHY that the connection isestablished, and the PHY returns OPEN accept. The router 40 forwards (atblock 176) the transmission to the selected protocol engine 46 a, 46 b.

Additionally, the application layer 50 may open a connection to transmitinformation to a target device by communicating the open request framesto one protocol engine 42 a, 42 b, using load balancing or some otherselecting technique, where the protocol engine 42 a, 42 b transport andport layers transmit the open connection frames to the router 40 todirect the link initialization to the appropriate link layer and PHYlayer.

FIG. 5 illustrates operations performed in the adaptor 12 to enable adevice driver 20 a, 20 b, 20 c to communicate information to a targetdevice through an adaptor 12 a, 12 b (FIG. 1). At block 200, a devicedriver 20 a, 20 b, 20 c transmits information to initiate communicationwith a connected device by sending (at block 202) information to aprotocol engine 46 a, 46 b. A device driver 20 a, 20 b, 20 c may performany operation to select a protocol engine to use. The protocol engine 46a, 46 b receiving the transmission forwards (at block 204) thetransmission to the router 40. If (at block 206) the protocol used bythe device driver 20 a, 20 b, 20 c is SATA or Fibre Channelpoint-to-point protocol, then the router 40 selects (at block 208) a PHY32 a, 32 b . . . 32 n connected to the target device (directly orindirectly through one or more expanders or a fabric) for transmissionand sends the transmission to the selected PHY. If (at block 206) theprotocol used by the device driver 20 a, 20 b, 20 c initiating thetransmission is SAS or Fibre Channel Arbitrated Loop, then the router 40selects (at block 210) a PHY 32 a, 32 b . . . 32 n to use to establishcommunication with the target device and add an entry to the routertable associating the protocol engine 42 a, 42 b forwarding thetransmission with the selected PHY, so that the indicated protocolengine and PHY are used for communications through that SAS or FibreChannel Arbitrated Loop connection. The router 40 then forwards (atblock 212) the open connection request through the selected PHY 32 a, 32b . . . 32 n to the target device.

Described embodiments provide techniques for allowing connections withdifferent storage interfaces that communicate at different transmissionspeeds and/or different transmission characteristics. In this way, asingle adaptor 12 may provide multiple connections for different storageinterfaces (storage interconnect architectures) that communicate usingdifferent transmission characteristics, such as transmitting atdifferent link speeds or including different protocol information in thetransmissions. For instance, the adaptor 12 may be included in anenclosure that is connected to multiple storage devices on a rack orprovides the connections for storage devices within the same enclosure.

Still further, with the described embodiments, there may be only oneserial to parallel conversion between the PHY layers 32 a, 32 b . . . 32n performing parallel-to-serial conversion and the protocol engines 42a, 42 b within the adaptor. In implementations where the expander islocated external to the adaptor, three parallel-to-serial conversionsmay be performed to communicate data from the connections to the router(serial to parallel), from the router in the expander to the adaptor(parallel to serial), and at the adaptor from the connection to theprotocol engine (serial to parallel). Certain described embodimentseliminate the need for two of these conversions by allowing the paralleldata to be transmitted directly from the router to the protocol enginesin the same adaptor component. Reducing the number of parallel to serialconversions and corresponding PLL tracking reduces data and bit errorsthat may be introduced by the frequency changes produced by the PLL inthe converters and may reduce latency delays caused by such additionalconversions.

Enclosure Architecture Supporting Multiple Protocols

FIG. 6 illustrates a storage enclosure 200 having a plurality of slots202 a and 202 b in which storage units 203 may be inserted. The storageunit may comprise a removable disk, such as a magnetic hard disk drive,tape cassette, optical disk, solid state disk, etc., may be inserted.Although only two slots are shown, any number of slots may be includedin the storage enclosure 200. The storage unit has a connector 205 tomate with one of the physical interfaces 204 a, 206 a and 204 b, 206 bon a backplane 208 of the enclosure 200 through one of the slots 202 a,202 b, respectively. A backplane comprises a circuit board includingconnectors, interfaces, slots into which components are plugged. Theslot 252 a, 252 b, 252 c comprises the space for receiving the storageunit 203 and may be delineated by a physical structure or boundaries,such as walls, guides, etc., or may comprise a space occupied by thestorage unit 203 that is not defined by any physical structures orboundaries. The physical interfaces 204 a, 206 a and 204 b, 206 bcorrespond to the physical interfaces 30 a, 30 b . . . 30 n in theadaptor. For instance, if the storage unit 203 is capable of mating withphysical interface 204 a, 204 b, then the user may rotate the storageunit 203 to allow the storage unit 203 to mate with that particularphysical interface 204 a, 204 b. If the storage unit 203 is capable ofmating with physical interface 206 a, 206 b, then the user may rotatethe storage unit 203 assembly 180 degrees to mate with physicalinterfaces 206 a, 206 b. In this way a single slot provides interfacesfor storage units whose physical interfaces have different physicalconfigurations, such as a different size dimensions, different interfacesizes, and different pin interconnect arrangements.

For instance, in certain embodiments, the physical interfaces 206 a and206 b may be capable of mating with a SATA/SAS physical interface andthe physical interfaces 204 a and 204 b may be capable of mating with aFibre Channel physical interface. In this way a single slot 202 a, 202 ballows mating with the storage unit having physical interfaces havingdifferent physical configurations. For instance, if the storage unit 203interface was designed to plug into a SAS/SATA interface, then the userwould rotate the storage unit 203 to interface with the physicalinterface, e.g., 204 a, supporting that interface, whereas if thestorage interface was designed to plug into a Fibre Channel interface,then the user would rotate the storage unit 203 to interface with thesupporting physical interface, e.g., 206 a.

In certain embodiments, the storage unit 203 may include only onephysical interface to mate with one physical interface, e.g., 204 a, 206a in one slot, e.g., 202 a.

FIG. 7 illustrates an embodiment of the architecture of the backplane258 of a storage enclosure 250, such as enclosure 200, having multipleslots 252 a, 252 b, 252 c (three are shown, but more or fewer may beprovided), where each slot has two physical interfaces 254 a, 256 a, 254b, 256 b, 254 c, 256 c. The physical interfaces 254 a, 254 b, 254 c and256 a, 256 b, 256 c may have different physical configurations, e.g.,size dimensions and pin arrangements, to support different storageinterconnect architectures, e.g., SATA/SAS and Fibre Channel. Anexpander 260 on the backplane 258 has multiple expander PHYs 262 a, 262b, 262 c. The expander PHYs 262 a, 262 b, 262 c may be organized intoone or more ports, where each port is assigned to have one or more PHYs.Further, one PHY 262 a, 262 b, 262 c may be coupled to each pair ofphysical interfaces 254 a, 256 a, 254 b, 256 b, 254 c, 256 c in eachslot 252 a, 252 b, 252 c. An expander function 266 routes informationfrom PHYs 262 a, 262 b, 262 c to destination PHYs 264 a, 264 b, 264 cfrom where the information is forwarded to an end device directly orthrough additional expanders. FIG. 7 shows the destination PHYs 264 a,264 b, 264 c connecting directly to the physical interfaces on anadaptor 280 in server 282.

In certain embodiments, a multidrop connector 266 a, 266 b, 266 cextends from the physical interface for each PHY 262 a, 262 b, 262 c toone of the slots 252 a, 252 b, 252 c, where each end on the multidropconnector 266 a, 266 b, 266 c is coupled to one of the interfaces 254 a,256 a; 254 b, 256 b; and 254 c, 256 c, respectively, in the slots 252 a,252 b, 252 c, respectively. A multidrop connector comprises acommunication line with multiple access points, where the access pointsmay comprise cable access points, etched path access points, etc. Inthis way, one multidrop connector provides the physical connection todifferent physical interfaces in one slot, where the different physicalinterfaces may have different physical dimensions and pin arrangements.To accommodate different physical interfaces, the multidrop connector268 a, 268 b, 268 c terminators includes different physical connectorsfor mating with the different storage interconnect physical interfacese.g., SAS/SATA, Fibre Channel, that may be on the storage unit 203,e.g., disk drive, inserted in the slot 252 a, 252 b, 252 c and mated tophysical interface 254 a, 256 a, 254 b, 256 b, 254 c, 256 c. Themultidrop connectors 266 a, 266 b, 266 c may comprise cables or pathsetched on a printed circuit board.

FIG. 8 illustrates components within an expander PHY 300, such asexpander PHYs 262 a, 262 b, 262 c, 264 a, 264 b, 264 c. An expander PHY300 may include a PHY layer 302 to perform PHY operations, and a PHYlink layer 304. Additionally, the PHY layer 302 may perform theoperations described with respect to the PHY layers 32 a, 32 b . . . 32n in FIG. 2 whose operations are described in FIG. 3. The expander PHYlayer 302 may include the capability to detect transmissioncharacteristics for different hardware interfaces, i.e., storageinterconnect architectures, e.g., SAS/SATA, Fibre Channel, etc., andforward information on the storage hardware interface to the link layer302, where the link layer 302 uses that information to access theaddress of the target storage device of the transmission to select theexpander PHY connected to the target device. This architecture for theexpander PHYs allows the expander to handle data transmitted fromdifferent storage interconnect architectures having differenttransmission characteristics.

The expander may further include a router to route a transmission fromone PHY to another PHY connecting to the target device or path to thetarget device. The expander router may further maintain a router tablethat associates PHYs with the address of the devices to which they areattached, so a transmission received on one PHY directed to an enddevice is routed to the PHY associated with that end device.

With respect to FIG. 7, the adaptor 280 in the server 282 may includethe same architecture as the adaptor 12 in FIG. 2, including theexpander 34 and protocol engine 42 a, 42 b architecture that operates asdescribed with respect to the embodiments of FIGS. 1, 2, 3, 4, and 5.The adaptor 280 receives data from the expander 260 in the storageenclosure 250 via connection 290 and then forward the transmission toone of the protocol engines 288 a, 288 b in the manner described above.Each physical interface 284 a, 284 b, 284 c on the server adaptor 280may connect to a different storage enclosure and each destination PHY264 a, 264 b, 264 on the backplane 258 expander 260 may be coupled to adifferent server, thereby allowing different servers to connect tomultiple storage enclosures and a storage enclosure to connect todifferent servers.

With the described embodiments, storage units, such as disk drives,having different connection interfaces may be inserted within the slots252 a, 252 b, 252 c (FIG. 7) on the backplane 258 by rotating theorientation of the storage unit assembly when inserting the storage unitin the slot. Further, the adaptor 280 may support transmissions from thebackplane 258 expander 260 using different storage interconnectarchitectures, such as SAS/SATA and Fibre Channel, by including thecomponents and performing the operations described above with respect toFIGS. 2, 3, 4, and 5. In this way, a single storage enclosure 250 mayallow for use of storage units, such as disk drives, having differentstorage interfaces, i.e., storage interconnect architectures, withdifferent physical interface arrangements, e.g., different dimensionsand pin arrangements. The use of the adaptor 280 and expander 260 on theenclosure backplane both supporting storage interconnect architectureshaving different transmission characteristics, e.g., link speed and dataformat, allows for communication with an enclosure capable of includingin its slots storage physical interfaces for different storageinterconnect architectures, e.g., Fibre Channel, SAS/SATA.

FIG. 9 illustrates a storage rack 310 including mounted servers 312 a,312 b and storage enclosures 314 a, 314 b. Only two of each are shown,but any number capable of being accommodated by the layout of the rackmay be included. In this example, each server 312 a, 312 b is connectedto each storage enclosure 314 a, 314 b. The storage enclosures 312 a,312 b may include a backplane 258 as described with respect to FIGS. 6and 7, and each server 312 a, 312 b may include an adaptor 280 asdescribed with respect to FIGS. 2 and 7 to support storage units usingdifferent storage interconnect architectures that require differentphysical interfaces and have different transmission characteristics.Each storage enclosure and server may include multiple adaptor cards toallow for additional connections.

FIG. 10 illustrates an alternative embodiment of an adaptor 320 that maybe substituted for the adaptor 280 in FIG. 7 connected to the storageenclosure 250. Adaptor 320 includes a plurality of ports 322, where eachport includes one or more PHYs 324, and where each PHY 324 has a PHYlayer 326, a link layer 328 and different protocol link layers, e.g., anSSP link layer 330 a, STP link layer 330 b, SMP link layer 330 c, and aFibre Channel Protocol link layer 330 d. In a port 322, all the PHYs inthat port share a link layer 332 and the transport layers, e.g., SSPtransport layer 334 a, Fibre Channel Protocol 334 b, STP transport layer334 c, and SMP transport layer 334 d. The PHY layer 326 and link layer328 in the embodiment of FIG. 10 performs the operations of the PHYlayers 32 a, 32 b . . . 32 n and link layer 36 as described with respectto FIGS. 2, 3, 4, and 54 to detect the transmission characteristics andcorresponding storage interconnect architecture therefrom and use thedetected storage interconnect architecture to process the packet anddetermine the link layer protocol, e.g., SSP, STP, SMP, Fibre ChannelProtocol to use. However, in the embodiment of FIG. 2, multiple PHYlayers in multiple ports may share the link layer, port layer andtransport layers, whereas in the embodiment of FIG. 10, each PHY has itsown link layer and each port has its own port layer and transportlayers, thereby providing greater redundancy of components. The STPprotocol can also uses SATA.

Described embodiments provide architectures to allow a single adaptorinterface to be used to interface with devices using different storageinterfaces, i.e., storage interconnect architectures, where some of thestorage interfaces use different and non-overlapping link speeds. Thisovercomes the situation where a single adaptor/controller, such a SASdevice, may not support storage interconnect architectures that havedifferent transmission characteristics, such as is the case where anadaptor supporting SAS/SATA may not support the Fibre Channel interfacebecause such an adaptor cannot detect data transmitted using the FibreChannel interface (storage interconnect architecture) and thus cannotload the necessary drivers in the operating system to support FibreChannel.

Additional Embodiment Details

The described embodiments may be implemented as a method, apparatus orarticle of manufacture using programming and/or engineering techniquesto produce software, firmware, hardware, or any combination thereof. Theterm “article of manufacture” and “circuitry” as used herein refers to astate machine, code or logic implemented in hardware logic (e.g., anintegrated circuit chip, Programmable Gate Array (PGA), ApplicationSpecific Integrated Circuit (ASIC), etc.) or a computer readable medium,such as magnetic storage medium (e.g., hard disk drives, floppy disks,tape, etc.), optical storage (CD-ROMs, optical disks, etc.), volatileand non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs,DRAMs, SRAMs, firmware, programmable logic, etc.). Code in the computerreadable medium is accessed and executed by a processor. When the codeor logic is executed by a processor, the circuitry would include themedium including the code or logic as well as the processor thatexecutes the code loaded from the medium. The code in which preferredembodiments are implemented may further be accessible through atransmission media or from a file server over a network. In such cases,the article of manufacture in which the code is implemented may comprisea transmission media, such as a network transmission line, wirelesstransmission media, signals propagating through space, radio waves,infrared signals, etc. Thus, the “article of manufacture” may comprisethe medium in which the code is embodied. Additionally, the “article ofmanufacture” may comprise a combination of hardware and softwarecomponents in which the code is embodied, processed, and executed. Ofcourse, those skilled in the art will recognize that many modificationsmay be made to this configuration, and that the article of manufacturemay comprise any information bearing medium known in the art.

Additionally, the expander, PHYs, and protocol engines may beimplemented in one or more integrated circuits on the adaptor or on themotherboard.

In the described embodiments, layers were shown as operating withinspecific components, such as the expander and protocol engines. Inalternative implementations, layers may be implemented in a mannerdifferent than shown. For instance, the link layer and link layerprotocols may be implemented with the protocol engines or the port layermay be implemented in the expander.

In the described embodiments, the protocol engines each support multipletransport protocols. In alternative embodiments, the protocol enginesmay support different transport protocols, so the expander 40 woulddirect communications for a particular protocol to that protocolsupporting the determined protocol.

In the described embodiments, transmitted information is received at anadaptor card from a remote device over a connection. In alternativeembodiments, the transmitted and received information processed by thetransport protocol layer or device driver may be received from aseparate process executing in the same computer in which the devicedriver and transport protocol driver execute.

In certain implementations, the device driver and network adaptorembodiments may be included in a computer system including a storagecontroller, such as a SCSI, Redundant Array of Independent Disk (RAID),etc., controller, that manages access to a non-volatile or volatilestorage device, such as a magnetic disk drive, tape media, optical disk,etc. In alternative implementations, the network adaptor embodiments maybe included in a system that does not include a storage controller, suchas certain hubs and switches.

In certain implementations, the adaptor may be configured to transmitdata across a cable connected to a port on the adaptor. In furtherembodiments, the adaptor may be configured to transmit data acrossetched paths on a printed circuit board. Alternatively, the adaptorembodiments may be configured to transmit data over a wireless networkor connection.

In described embodiments, the storage interfaces supported by theadaptors comprised SATA, SAS and Fibre Channel. In additionalembodiments, other storage interfaces may be supported. Additionally,the adaptor was described as supporting certain transport protocols,e.g. SSP, Fibre Channel Protocol, STP, and SMP. In furtherimplementations, the adaptor may support additional transport protocolsused for transmissions with the supported storage interfaces. Thesupported storage interfaces may transmit using different transmissioncharacteristics, e.g., different link speeds and different protocolinformation included with the transmission. Further, the physicalinterfaces may have different physical configurations, i.e., thearrangement and number of pins and other physical interconnectors, whenthe different supported storage interconnect architectures use differentphysical configurations.

The adaptor 12 may be implemented on a network card, such as aPeripheral Component Interconnect (PCI) card or some other I/O card, oron integrated circuit components mounted on a system motherboard orbackplane.

In described embodiments, the interfaces in the slot extend along thevertical length of the slot and are in a parallel orientation withrespect to each other. In alternative embodiments, the two interfacesmay be oriented in different ways with respect to each other and theslot depending on the corresponding interface on the storage unitassembly. Further, in additional implementations more than two physicalinterfaces may be included in the slot for the different protocolssupported by the adaptor.

The illustrated logic of FIGS. 3, 4, and 5 show certain events occurringin a certain order. In alternative embodiments, certain operations maybe performed in a different order, modified or removed. Moreover,operations may be added to the above described logic and still conformto the described embodiments. Further, operations described herein mayoccur sequentially or certain operations may be processed in parallel.Yet further, operations may be performed by a single processing unit orby distributed processing units.

FIG. 11 illustrates one implementation of a computer architecture 400 ofthe storage enclosures and servers in FIGS. 6 and 8. The architecture400 may include a processor 402 (e.g., a microprocessor), a memory 404(e.g., a volatile memory device), and storage 406 (e.g., a non-volatilestorage, such as magnetic disk drives, optical disk drives, a tapedrive, etc.). The storage 406 may comprise an internal storage device oran attached or network accessible storage. Programs in the storage 406are loaded into the memory 404 and executed by the processor 402 in amanner known in the art. The architecture further includes an adaptor asdescribed above with respect to FIGS. 1-7 to enable a point-to-pointconnection with an end device, such as a disk drive assembly. Asdiscussed, certain of the devices may have multiple network cards. Aninput device 410 is used to provide user input to the processor 402, andmay include a keyboard, mouse, pen-stylus, microphone, touch sensitivedisplay screen, or any other activation or input mechanism known in theart. An output device 412 is capable of rendering informationtransmitted from the processor 402, or other component, such as adisplay monitor, printer, storage, etc.

The foregoing description of various embodiments has been presented forthe purposes of illustration and description. It is not intended to beexhaustive or to limit the embodiments to the precise form disclosed.Many modifications and variations are possible in light of the aboveteaching

1. A method for using an expander on a backplane to interface storageunits with an external device coupled to the expander, comprising theexpander performing: communicating with one storage unit interfacingwith a first physical interface, coupled to the expander, in a slot of astorage enclosure when the storage unit is received in a firstorientation with respect to the slot; communicating with one storageunit interfacing with a second physical interface, coupled to theexpander, in the slot when the storage unit is received in a secondorientation with respect to the slot, wherein the first and secondphysical interface supports different storage interconnect architecturesreceiving a transmission from one storage unit in one slot coupled tothe expander; maintaining information on storage interconnectarchitectures and transmission characteristics used for the storageinterconnect architectures, wherein the storage interconnectarchitectures have transmission characteristics; determining thetransmission characteristic of the received transmission; determiningfrom the information the storage interconnect architecture associatedwith the determined transmission characteristics; using the informationon the determined storage interconnect architecture to process thetransmission and determine a transport layer for the receivedtransmission, wherein there is one transport layer for each supportedtransport protocol; and forwarding the transmission to the determinedtransport layer.
 2. The method of claim 1, wherein the different storageinterconnect architectures supported by the physical interfaces havedifferent transmission characteristics.
 3. The method of claim 1,wherein the two physical interfaces in the slots have different physicalconfigurations.
 4. The method of claim 1, wherein the physicalinterfaces each extend along a vertical length of the slot and areparallel to each other.
 5. The method of claim 1, wherein the supportedstorage interconnect architectures comprise SATA, SAS, and Fibre Channeland wherein the supported transport protocols comprise SSP, FibreChannel Protocol, STP, SMP, and SATA.
 6. The method of claim 1, whereinin the slots, one physical interface complies with the SATA/SAS storageinterconnect architecture and another physical interface complies withthe Fibre Channel storage interconnect architecture.
 7. The method ofclaim 1, wherein at least one physical interface supports multiplestorage interconnect architectures.
 8. The method of claim 1, furthercomprising: connecting a multidrop connection having a first end incommunication with a server and a second and third ends coupled to thetwo interfaces in one slot.